Method and arrangement for detecting the spatial form of an object

ABSTRACT

The invention relates to a method of detecting the 3D shape of objects, in which the object is pressed into a plastically deformable mass provided with markers adapted to be evaluated photogrammetrically. After removal of the object from the marked mass, a plurality of photogrammetric images are taken from different views, and the images are evaluated photogrammetrically, the 3D shape of the object being calculated using an association of the markers corresponding in the images. The invention further relates to a corresponding arrangement for carrying out the method and is particularly suited for the detection of the 3D shape of objects whose 3D shape changes under load. One example of this would be the measurement of human feet.

[0001] The invention relates to a method and an arrangement for thedetection of the 3D shape of an object such as a human foot.

[0002] Detection of the three-dimensional shape (3D shape) of bodies andbody parts is an important preliminary stage for the production andselection of products whose shape needs to be adapted to the shape ofthe bodies or body parts. Examples of such products are shoes, footbedsor arch supports, prostheses, ortheses, or articles of clothing.

[0003] It is known to detect body parts, e.g., optically, using lasertriangulation methods, methods based on stripe projection, or by opticaldetection of a sequence of silhouette outlines. In contrast to thesemethods, which are relatively involved because of their mechanically andoptically accurately constructed and calibrated systems, European PatentEP 0 760 622, “Sensing Process and Arrangement for the Three-DimensionalShape in Space of Bodies or Body Parts”, discloses a very simplesolution which is based on the photogrammetric evaluation of images ofthe body part which are obtained from non-calibrated imaging positions.To this end, the body part is clad in an elastic envelope provided withmarkers adapted to be evaluated photogrammetrically.

[0004] German Patent Application No. 100 25 922.7 “Automatischephotogrammetrische Digitalisierung von Körpem und Objekten” (Automaticphotogrammetric digitization of bodies and objects) further discloseshow by a special marking of the background of such markers, i.e. byapplying area markers, a referencing of the markers (point markers),i.e. an association of pairs of identical point markers in the differentoverlapping images can be facilitated and automated.

[0005] All of these aforementioned processes digitize the body whenunloaded. In particular in the case of soft tissue, however, the 3Dshape of a loaded body differs greatly from the 3D shape of an unloadedbody.

[0006] For numerous products intended to be adapted to the body, such asa footbed, a seat shaped to fit a person's anatomy, etc., the spatialcoordinates of the body part deformed under load are required ratherthan those of the unloaded and non-deformed body part.

[0007] When measuring the sole of the foot for manufacturing a fittinginsole or arch support or a footbed made to fit, for example, theconventional procedure involves the person placing the foot into aplastically deformable mass, and obtaining an impression from thisdeformed mass using traditional molding techniques, such impressionconstituting the starting position for preparing the arch support. Thisis an awkward, lengthy and costly process which could be considerablyimproved by an automatic digitization of the sole of the foot underload.

[0008] It is therefore the object of the invention to provide a simplemethod and a simple arrangement for the detection of the 3D shape of anobject, which solve the problems explained above appearing in previousmethods and arrangements and which are more particularly suitable forsimple detection of the 3D shape of objects that need to be measuredunder load.

[0009] This object is attained by a method of detecting the 3D shape ofan object, wherein a plastically deformable mass is provided which hasat least one surface portion provided with markers adapted to beevaluated photogrammetrically, the object is impressed on the surfaceportion into the mass such that the mass deforms corresponding to the 3Dshape of the object, the object is removed from the deformed mass, aplurality of images are taken of the deformed mass provided with themarkers from respective different views, and the 3D shape of the objectis determined from the images by means of a photogrammetric process.

[0010] The object is further attained by an arrangement for detectingthe 3D shape of an object by photogrammetry, comprising an imagingsystem for obtaining photogrammetric images and a system forphotogrammetrically evaluating the images and for determining the 3Dshape, which is characterized in that it further comprises a plasticallydeformable mass which has at least one surface portion provided withmarkers adapted to be evaluated photogrammetrically.

[0011] In accordance with the invention the optical detection of the 3Dshape of an impression material loaded and deformed by a body part isachieved in that the deformable impression material is characterized byhigh-contrast markers adapted to be evaluated photogranunetrically. Inparticular, in accordance with the invention these markings may consistof inclusions whose color differs from the color of the deformableimpression mass or of differently colored local colorations of theimpression material. By detecting a series of overlappingtwo-dimensional images (2D images) from unknown imaging positions of thedeformed and marked impression mass, the three-dimensional shape (3Dshape) of the impression and, hence, the 3D shape of the body partdeformed under load can be determined from these 2D images in a simpleway using known methods of photogrammetry.

[0012] Advantageous further developments of the invention arecharacterized in the dependent claims.

[0013] Further features and advantages of the invention will be apparentfrom the following description of an embodiment with reference to thedrawings, in which:

[0014]FIG. 1 shows a plastically deformable impression mass used in oneembodiment of the invention and provided with markers adapted to beevaluated photogrammetrically, the impression mass being located in animpression frame and being used to prepare an impression of the sole ofthe foot under load;

[0015]FIG. 2 illustrates a schematic arrangement of cameras thepositions of which are not known and which are used to take overlapping2D images of the impression which contain the markers and the positionsthereof in relation to the 2D coordinates of these images; and

[0016]FIG. 3 shows, as an example, a pseudo 3D representation of markersreproducing the three-dimensional shape of the footbed resulting fromthe impression in FIG. 2.

[0017] The method of the invention will now be described by way ofexample in connection with a typical application, the detection of thesole of a foot under load for manufacturing a footbed that fitsanatomically.

[0018] This example is, of course, by no means the only possibleapplication of the method in accordance with the invention. Forinstance, the detection of a 3D impression of the buttocks formanufacturing anatomically adapted seating surfaces would be anotherexample where the detection of the 3D geometry of a body deformed byload is important. Many other applications are conceivable.

[0019] Within the scope of the idea of the invention, the terms “object”or “body” are by no means limited to human or animal bodies only, butfurther cover any physical bodies such as, e.g., mechanical models,antique statues, sculptures or similar three-dimensional spatial shapes.

[0020] As shown in FIG. 1, within the frame of a mold 1 there is held aplastically deformable mass 2 into which a person places his/her foot 3under load. At least a portion 7 of the surface of the mass 2 isprovided with markers 4 that are adapted to be evaluatedphotogrammetrically.

[0021] The mass 2 consists of a base material of a single color, intowhich a number of markers 4 adapted to be evaluated photogrammetricallyare let in, whose contrasting color stands out against the color of thebase material. These markers may consist of embedded pins of a differentcolor, for example, which are made of the same material as the basematerial and are let into the base material. The pins may becylindrical, for example. The base material may consist of commonlyavailable deformable materials, for example, such as plaster, wax,alginate, or elastomers (e.g., silicones).

[0022] In a preferred embodiment a plastically deformable material isselected which may be reversibly deformed; after deformation caused bythe pressure exerted by the object and after termination of thephotogrammetric evaluation of the impression, the material may bereturned to its original form under the influence of energy, heat,light, or mechanical energy. Those of ordinary skill in the art arefamiliar with prior art materials of this type. When such a material isused, the mass provided with photogrammetric markers may be reused asoften as desired.

[0023] According to the invention, for a simple photogrammetricevaluation the background around these markers may also be marked byzones of a different color. This results in a marking on two levels,namely consisting of point markers and area markers, each of the areamarkers comprising a plurality of point markers, forming a background ofthe point markers, and having a characteristic optical configuration.

[0024] The impression mass 2 will deform under the load of the footplaced onto the surface portion 7 of the mass 2. The markers 4 which arevisible from outside will be shifted accordingly both laterally and inregard to the depth thereof.

[0025] After the foot 3 has been removed from the mass 2, images aretaken of the impression 6 that is left, illustrated in FIG. 2, from anumber of overlapping imaging positions, using an imaging system whichmay include a camera or, as shown in FIG. 2, a plurality of cameras 5,and a number of 2D images are prepared, in which the markers 4 occupydifferent XY positions.

[0026] A person of ordinary skill in the art of photogrammetry andshort-range photogrammetry knows how to evaluate such images forobtaining the 3D coordinates, which is why this will not be explained ingreater detail herein.

[0027] In the photogrammetric method, first an image processing of theimages taken is carried out, in which the markers respectivelycorresponding in the images are associated with one another. Based onthe marker association, the 3D shape of the impression and, from this,the 3D shape of the object is then determined using known mathematicalmethods of photogrammetry.

[0028] Where point and area markers are employed, as set forth above, inthe image processing of the images first the area markers respectivelycorresponding in the images are associated with one another using theircharacteristic optical configurations. Then the point markers comprisedby the area markers and respectively corresponding in the images areassociated with one another with the aid of the area marker association.Then, using the point marker association, the 3D shape of the object isdetermined by means of a photogrammetric evaluation process.

[0029] In a preferred embodiment, a measuring rod may be mounted on thatside of the mold holding the plastically deformable mass which isopposite to the photogrammetric imaging system, the measuring rod havingarranged thereon at least two photogrammetric markers at a knowndistance from each other. Such a measuring rod may consist, e.g., of arigid, straight rod or plate having the markers applied thereon. Themeasuring rod may of course also be arranged separately beside the mold.The photogrammetric images are then taken in such a way that at leasttwo images of the impression bearing the markers are taken from anydesired positions, with the measuring rod being also included in theimages at the same time. This allows the 3D shape of the impression orof the object impressed to be calculated later in absolute values withthe aid of the unit of length given by the measuring rod when the imagesare evaluated photogrammetrically.

[0030]FIG. 3 shows, by way of example, the pseudo 3D representation ofthe markers reflecting the 3D shape of the footbed and thus permittingan automatic production of a footbed using, e.g., a triaxial cutter.

[0031] In accordance with the invention, the impression material mayalso be marked so as to make it suitable for photogrammetric evaluationin that it is covered, prior to being loaded, by a marked elasticenvelope, as described in EP 0 760 622, which may consist of a textilematerial or a synthetic material, for instance. Preferably, thisenvelope is intended to adhere to the impression material, so that thedeformation of the mass is transferred as accurately as possible to thedeformation of the elastic envelope. Use of such an envelope solves theproblem of hygiene that is created when there is a direct skin contactwith the impression mass.

[0032] As a disposable article, this envelope obviates the necessity ofa costly disinfection of the impression material following each use.

[0033] The method according to the invention may be transferred to manyother impression or molding applications, of which only a few will belisted as an example:

[0034] a) the manufacture of anatomically fitting seating surfaces byuse of a marked, flat, plastic mass into which the customer sits himselfdown, thus leaving an impression of his buttocks deformed by load;

[0035] b) the manufacture of spectacle frames that fit well in the nosearea by an impression of the region of the nasal root, using animpression material marked in accordance with the invention;

[0036] c) the manufacture of copies of antique objects by producing animpression in a plastic material marked in accordance with theinvention.

[0037] The terms “detection of the 3D shape of an object” and“determination of the 3D shape of an object” as used in the presentspecification and in the claims are intended to be construed in a senseso as to include detection of the negative shape of the object, i.e. theimpression of an object, since the positive shape of the object alsoimplicitly results therefrom, i.e. as a result, the measurements of theobject are provided. These terms are further intended to relate also tothe detection of the 3D shape of a part of an object, such as, e.g., ofa foot as part of the human body.

1. A method of detecting the 3D shape of an object (3), wherein aplastically deformable mass (2) is provided which has at least onesurface portion (7) provided with markers (4) adapted to be evaluatedphotogrammetrically, the object is impressed on the surface portion (7)into the mass (2) such that the mass (2) deforms corresponding to the 3Dshape of the object (3), the object (3) is removed from the deformedmass (2), a plurality of images are taken of the deformed mass (2)provided with the markers (4) from respective different views, and the3D shape of the object (3) is determined from the images by means of aphotogrammetric process.
 2. The method as claimed in claim 1, wherein inthe photogrammetric process an image processing of the images isperformed, in which the markers (4) respectively corresponding in theimages are associated with one another, and the 3D shape of the object(3) is determined using the marker association.
 3. The method as claimedin claim 1 or 2, wherein the provision of the plastically deformablemass which has at least one surface portion provided with markersadapted to be evaluated photogrammetrically is effected by providing aplastically deformable mass having bodies enclosed therein which have acolor that is different from the color of the mass and which are visiblefrom outside in the area of the surface portion (7).
 4. The method asclaimed in claim 1 or 2, wherein the provision of the plasticallydeformable mass which has at least one surface portion provided withmarkers adapted to be evaluated photogrammetrically is effected byapplying markers of a color that is different from the color of the massonto the surface portion of the plastically deformable mass.
 5. Themethod as claimed in claim 1 or 2, wherein the provision of theplastically deformable mass which has at least one surface portionprovided with markers adapted to be evaluated photogrammetrically iseffected by placing an elastic envelope provided with markers on thesurface portion of the plastically deformable mass.
 6. The method asclaimed in any of the preceding claims, wherein the markers consist ofpoint markers and area markers, each of the area markers comprising aplurality of point markers, forming a background of the point markers,and having a characteristic optical configuration.
 7. The method asclaimed in claim 6, wherein in the image processing of the images firstthe area markers respectively corresponding in the images are associatedwith one another using their characteristic optical configurations, andthen the point markers comprised by the area markers and respectivelycorresponding in the images are associated with one another with the aidof the area marker association, and using the point marker association,the 3D shape of the object is determined by means of a photogrammetricevaluation process.
 8. The method as claimed in any of the precedingclaims, wherein the plastically deformable mass is provided in a mold(1) which serves to hold the mass.
 9. The method as claimed in any ofthe preceding claims, wherein the object (3) is a body part of a person,whose 3D shape is to be measured under load.
 10. The method as claimedin claim 9, wherein the object (3) is a foot.
 11. The method as claimedin any of claims 1 to 8, wherein the object is a statue.
 12. The methodas claimed in any of the preceding claims, wherein the plasticallydeformable mass provided is a reversibly deformable mass which, once theimages have been taken, is returned to its original form which it hadtaken prior to the impression by the object.
 13. The method as claimedin claim 12, wherein the reversibly deformable mass is returned to itsoriginal form under the influence of energy, heat, light, or mechanicalenergy.
 14. The method as claimed in any of the preceding claims,wherein prior to obtaining the images, a measuring rod is arrangedbeside the surface portion of the deformable mass bearing the markersadapted to be evaluated photogrammetrically, the measuring rod havingarranged thereon at least two markers at a known distance from eachother, adapted to be evaluated photogrammetrically.
 15. The method asclaimed in claim 14, wherein at least two of the images of the massprovided with markers are taken such that the measuring rod is includedin the image taken.
 16. An arrangement for detecting the 3D shape of anobject by photogrammetry, comprising an imaging system (5) for obtainingphotogrammetric images and a system for photogrammetrically evaluatingthe images and for determining the 3D shape of the object, characterizedin that the arrangement further comprises a plastically deformable mass(2) which has at least one surface portion (7) provided with markers (4)adapted to be evaluated photogrammetrically.
 17. The arrangement asclaimed in claim 16, wherein bodies of a color different from the colorof the mass are provided to serve as markers, the bodies being enclosedin the mass and being visible from outside.
 18. The arrangement asclaimed in claim 16, wherein color markings serving as markers areprovided on the surface portion of the mass and have a color that isdifferent from the color of the mass.
 19. The arrangement as claimed inclaim 16, wherein an elastic envelope is provided having markers appliedthereon, which is pulled over the mass in the area of the surfaceportion and adheres to the mass.
 20. The arrangement as claimed in anyof claims 16 to 19, wherein further provided is a mold (1) to hold themass.
 21. The arrangement as claimed in any of claims 16 to 20, whereinthe plastically deformable mass is a reversibly deformable mass.
 22. Thearrangement as claimed in claim 21, wherein the reversibly deformablemass is made such that it can be returned to its original form under theinfluence of energy, heat, light, or mechanical energy.
 23. Thearrangement as claimed in any of claims 16 to 22, wherein a measuringrod is provided which bears at least two markers adapted to be evaluatedphotogrammetrically, arranged at a known distance from each other, themeasuring rod being arranged beside the surface portion of thedeformable mass.
 24. The arrangement as claimed in claim 20 and 23,wherein the measuring rod is connected to the mold holding the mass.